Skip to content

fix: sync through empty epochs in range sync#9417

Open
wemeetagain wants to merge 5 commits into
unstablefrom
cayman/fix-empty-epoch-sync
Open

fix: sync through empty epochs in range sync#9417
wemeetagain wants to merge 5 commits into
unstablefrom
cayman/fix-empty-epoch-sync

Conversation

@wemeetagain
Copy link
Copy Markdown
Member

@wemeetagain wemeetagain commented May 28, 2026

Motivation

A peer's correct response for an epoch in which every slot was skipped is zero
blocks. Range sync rejected that response, so the node could not sync past an
empty epoch during periods of poor chain liveness.

Two interacting defects, both introduced during the fulu/PeerDAS sync work:

  1. validateBlockByRangeResponse threw MISSING_BLOCKS_RESPONSE on a 0-block
    response. In sendBatch this became a downloadingError; since every peer
    returns empty for a genuinely empty epoch, all retries failed and after
    MAX_BATCH_DOWNLOAD_ATTEMPTS the batch threw MAX_DOWNLOAD_ATTEMPTS and the
    whole SyncChain errored out. A second copy of the throw in validateResponses
    fired for post-Deneb epochs that also carry a blobs/columns request.

  2. The MAX_LOOK_AHEAD_EPOCHS guard in includeNextBatch counted batches in the
    AwaitingValidation state. Empty batches reach AwaitingValidation but never
    call advanceChain (only non-empty batches do), so lastEpochWithProcessBlocks
    never advances. A run of empty epochs longer than MAX_LOOK_AHEAD_EPOCHS (2)
    filled the look-ahead window, includeNextBatch returned null, and the chain
    stalled with nothing left to download or process. This deadlock is why the
    naive "return [] instead of throw" fix had previously been reverted.

Description

  • validateBlockByRangeResponse returns an empty result with a
    MISSING_BLOCKS_RESPONSE warning instead of throwing. Whether an epoch is
    genuinely empty is enforced at the chain level: the empty batch is held in
    AwaitingValidation and only confirmed once a later batch imports a block, so
    a peer cannot stall sync by falsely claiming an epoch is empty.
  • validateResponses skips data-sidecar validation when there are no blocks in
    the data request's slot range (parent-by-root columns and envelopes still
    run).
  • includeNextBatch's look-ahead guard ignores AwaitingValidation batches,
    mirroring the BATCH_BUFFER_SIZE carve-out.

Archeology

  • downloadByRange.ts was introduced in feat: refactor block input #8200 (feat: refactor block input,
    2025-09-18) -- the fulu block-input/PeerDAS sync rewrite that deleted the old
    Deneb-era beaconBlocksMaybeBlobsByRange.ts downloader, which had tolerated
    empty epochs. The empty-blocks throw was then cemented in fix: refactor validateColumnsByRangeResponse #8482 (fix:
    refactor validateColumnsByRangeResponse, 2025-10-27, b992c32).
  • The MAX_LOOK_AHEAD_EPOCHS guard was added 2025-08-11 (38889e2) during
    PeerDAS sync work. The original 2021 BATCH_BUFFER_SIZE check directly above it
    (f46b63f, dapplion) deliberately excluded AwaitingValidation batches "to
    prevent stalling sync if the current processing window is contained in a long
    range of skip slots." The new guard did not replicate that carve-out,
    re-creating the exact stall the 2021 code was written to avoid.

AI Assistance Disclosure

  • claude assistance

A peer's correct response for an epoch in which every slot was skipped is zero
blocks. Range sync rejected that response, so the node could not sync past an
empty epoch during periods of poor chain liveness.

Two interacting defects, both introduced during the fulu/PeerDAS sync work:

1. validateBlockByRangeResponse threw MISSING_BLOCKS_RESPONSE on a 0-block
   response. In sendBatch this became a downloadingError; since every peer
   returns empty for a genuinely empty epoch, all retries failed and after
   MAX_BATCH_DOWNLOAD_ATTEMPTS the batch threw MAX_DOWNLOAD_ATTEMPTS and the
   whole SyncChain errored out. A second copy of the throw in validateResponses
   fired for post-Deneb epochs that also carry a blobs/columns request.

2. The MAX_LOOK_AHEAD_EPOCHS guard in includeNextBatch counted batches in the
   AwaitingValidation state. Empty batches reach AwaitingValidation but never
   call advanceChain (only non-empty batches do), so lastEpochWithProcessBlocks
   never advances. A run of empty epochs longer than MAX_LOOK_AHEAD_EPOCHS (2)
   filled the look-ahead window, includeNextBatch returned null, and the chain
   stalled with nothing left to download or process. This deadlock is why the
   naive "return [] instead of throw" fix had previously been reverted.

Archeology:
- downloadByRange.ts was introduced in #8200 (feat: refactor block input,
  2025-09-18) -- the fulu block-input/PeerDAS sync rewrite that deleted the old
  Deneb-era beaconBlocksMaybeBlobsByRange.ts downloader, which had tolerated
  empty epochs. The empty-blocks throw was then cemented in #8482 (fix:
  refactor validateColumnsByRangeResponse, 2025-10-27, b992c32).
- The MAX_LOOK_AHEAD_EPOCHS guard was added 2025-08-11 (38889e2) during
  PeerDAS sync work. The original 2021 BATCH_BUFFER_SIZE check directly above it
  (f46b63f, dapplion) deliberately excluded AwaitingValidation batches "to
  prevent stalling sync if the current processing window is contained in a long
  range of skip slots." The new guard did not replicate that carve-out,
  re-creating the exact stall the 2021 code was written to avoid.

Fix:
- validateBlockByRangeResponse returns an empty result with a
  MISSING_BLOCKS_RESPONSE warning instead of throwing. Whether an epoch is
  genuinely empty is enforced at the chain level: the empty batch is held in
  AwaitingValidation and only confirmed once a later batch imports a block, so
  a peer cannot stall sync by falsely claiming an epoch is empty.
- validateResponses skips data-sidecar validation when there are no blocks in
  the data request's slot range (parent-by-root columns and envelopes still
  run).
- includeNextBatch's look-ahead guard ignores AwaitingValidation batches,
  mirroring the BATCH_BUFFER_SIZE carve-out.

Tests:
- validateBlockByRangeResponse accepts an empty response during chain liveness
  issues.
- validateResponses accepts an empty epoch that carries a data request.
- chain.test.ts gains a "3 epochs of skipped slots" case -- a run longer than
  MAX_LOOK_AHEAD_EPOCHS -- which previously deadlocked.

Refs: #8147
validateResponses had three separate "nothing to validate here, fall through to
parent-by-root columns + envelope validation" paths, with that parent + envelope
tail duplicated between the no-data-request branch and the main path. The
empty-epoch fix made the duplication more pronounced by adding a third skip path.

Collapse them: gate getBlocksForDataValidation on `dataRequest` (defaulting to an
empty list when absent) and let the single shared tail handle parent-by-root
columns and envelopes for every case. This deletes the no-data-request early
return entirely, and flattens data-sidecar validation from an outer
`if (blocksForDataValidation.length > 0)` wrapper into per-request guards.

Behavior-preserving: an envelopes-only / parent-payload-only request and an empty
epoch now flow through the same path that already validated them, minus the
duplicated branch. Net -20 lines.

Adds a characterization test pinning the no-data-request (envelopes-only) path,
which previously had no direct unit coverage.
@wemeetagain wemeetagain requested a review from a team as a code owner May 28, 2026 20:15
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses sync deadlocks during periods of poor chain liveness by allowing the sync chain to process empty epochs (epochs with zero blocks) without stalling or throwing errors. It updates the look-ahead window check to only count pending batches, preventing empty batches in AwaitingValidation from filling the window. Additionally, it refactors the response validation to return warnings instead of throwing errors when zero blocks are returned for an epoch, and adds corresponding regression tests. A review comment suggests simplifying a redundant ternary operator in validateResponses when passing validated blocks to getBlocksForDataValidation.

Comment thread packages/beacon-node/src/sync/utils/downloadByRange.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fe749a76a7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/beacon-node/src/sync/utils/downloadByRange.ts
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 443e8ce Previous: 2e689b4 Ratio
send data - 1000 16384B messages 71.664 ms/op 22.447 ms/op 3.19
Full benchmark results
Benchmark suite Current: 443e8ce Previous: 2e689b4 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.2718 ms/op 1.0785 ms/op 1.18
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 40.552 us/op 43.636 us/op 0.93
BLS verify - blst 723.50 us/op 654.89 us/op 1.10
BLS verifyMultipleSignatures 3 - blst 1.3745 ms/op 1.4150 ms/op 0.97
BLS verifyMultipleSignatures 8 - blst 2.1820 ms/op 2.4208 ms/op 0.90
BLS verifyMultipleSignatures 32 - blst 6.8902 ms/op 6.9483 ms/op 0.99
BLS verifyMultipleSignatures 64 - blst 13.152 ms/op 13.124 ms/op 1.00
BLS verifyMultipleSignatures 128 - blst 25.853 ms/op 25.597 ms/op 1.01
BLS deserializing 10000 signatures 650.62 ms/op 648.43 ms/op 1.00
BLS deserializing 100000 signatures 6.5731 s/op 6.4719 s/op 1.02
BLS verifyMultipleSignatures - same message - 3 - blst 823.01 us/op 770.43 us/op 1.07
BLS verifyMultipleSignatures - same message - 8 - blst 967.51 us/op 899.24 us/op 1.08
BLS verifyMultipleSignatures - same message - 32 - blst 1.5360 ms/op 1.4809 ms/op 1.04
BLS verifyMultipleSignatures - same message - 64 - blst 2.3849 ms/op 2.3849 ms/op 1.00
BLS verifyMultipleSignatures - same message - 128 - blst 4.0682 ms/op 4.1197 ms/op 0.99
BLS aggregatePubkeys 32 - blst 17.871 us/op 18.066 us/op 0.99
BLS aggregatePubkeys 128 - blst 63.141 us/op 64.395 us/op 0.98
getSlashingsAndExits - default max 50.545 us/op 61.716 us/op 0.82
getSlashingsAndExits - 2k 360.14 us/op 409.46 us/op 0.88
proposeBlockBody type=full, size=empty 1.0103 ms/op 874.92 us/op 1.15
isKnown best case - 1 super set check 191.00 ns/op 172.00 ns/op 1.11
isKnown normal case - 2 super set checks 165.00 ns/op 169.00 ns/op 0.98
isKnown worse case - 16 super set checks 167.00 ns/op 170.00 ns/op 0.98
validate api signedAggregateAndProof - struct 1.5379 ms/op 1.5184 ms/op 1.01
validate gossip signedAggregateAndProof - struct 1.5315 ms/op 1.5206 ms/op 1.01
batch validate gossip attestation - vc 640000 - chunk 32 109.61 us/op 109.27 us/op 1.00
batch validate gossip attestation - vc 640000 - chunk 64 95.525 us/op 99.336 us/op 0.96
batch validate gossip attestation - vc 640000 - chunk 128 90.638 us/op 92.500 us/op 0.98
batch validate gossip attestation - vc 640000 - chunk 256 87.729 us/op 89.211 us/op 0.98
bytes32 toHexString 287.00 ns/op 287.00 ns/op 1.00
bytes32 Buffer.toString(hex) 172.00 ns/op 165.00 ns/op 1.04
bytes32 Buffer.toString(hex) from Uint8Array 242.00 ns/op 234.00 ns/op 1.03
bytes32 Buffer.toString(hex) + 0x 171.00 ns/op 159.00 ns/op 1.08
Return object 10000 times 0.21110 ns/op 0.21370 ns/op 0.99
Throw Error 10000 times 3.3786 us/op 3.3065 us/op 1.02
toHex 96.228 ns/op 88.527 ns/op 1.09
Buffer.from 83.394 ns/op 86.095 ns/op 0.97
shared Buffer 53.178 ns/op 52.477 ns/op 1.01
fastMsgIdFn sha256 / 200 bytes 1.4950 us/op 1.4730 us/op 1.01
fastMsgIdFn h32 xxhash / 200 bytes 172.00 ns/op 158.00 ns/op 1.09
fastMsgIdFn h64 xxhash / 200 bytes 221.00 ns/op 205.00 ns/op 1.08
fastMsgIdFn sha256 / 1000 bytes 4.7810 us/op 4.8000 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 261.00 ns/op 249.00 ns/op 1.05
fastMsgIdFn h64 xxhash / 1000 bytes 270.00 ns/op 249.00 ns/op 1.08
fastMsgIdFn sha256 / 10000 bytes 42.671 us/op 43.298 us/op 0.99
fastMsgIdFn h32 xxhash / 10000 bytes 1.2940 us/op 1.2640 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 845.00 ns/op 804.00 ns/op 1.05
send data - 1000 256B messages 4.8999 ms/op 4.9460 ms/op 0.99
send data - 1000 512B messages 4.5865 ms/op 4.9954 ms/op 0.92
send data - 1000 1024B messages 5.7566 ms/op 5.1021 ms/op 1.13
send data - 1000 1200B messages 5.3470 ms/op 5.5398 ms/op 0.97
send data - 1000 2048B messages 5.5920 ms/op 6.1396 ms/op 0.91
send data - 1000 4096B messages 6.2755 ms/op 7.0511 ms/op 0.89
send data - 1000 16384B messages 71.664 ms/op 22.447 ms/op 3.19
send data - 1000 65536B messages 243.03 ms/op 120.61 ms/op 2.01
enrSubnets - fastDeserialize 64 bits 740.00 ns/op 733.00 ns/op 1.01
enrSubnets - ssz BitVector 64 bits 263.00 ns/op 251.00 ns/op 1.05
enrSubnets - fastDeserialize 4 bits 105.00 ns/op 102.00 ns/op 1.03
enrSubnets - ssz BitVector 4 bits 254.00 ns/op 247.00 ns/op 1.03
prioritizePeers score -10:0 att 32-0.1 sync 2-0 199.33 us/op 207.10 us/op 0.96
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 243.08 us/op 425.28 us/op 0.57
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 341.67 us/op 358.49 us/op 0.95
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 600.28 us/op 611.20 us/op 0.98
prioritizePeers score 0:0 att 64-1 sync 4-1 699.43 us/op 695.35 us/op 1.01
array of 16000 items push then shift 1.2633 us/op 1.2197 us/op 1.04
LinkedList of 16000 items push then shift 7.5950 ns/op 7.5140 ns/op 1.01
array of 16000 items push then pop 74.641 ns/op 67.554 ns/op 1.10
LinkedList of 16000 items push then pop 6.1210 ns/op 5.9840 ns/op 1.02
array of 24000 items push then shift 1.8758 us/op 1.7932 us/op 1.05
LinkedList of 24000 items push then shift 7.5490 ns/op 7.4560 ns/op 1.01
array of 24000 items push then pop 108.91 ns/op 98.161 ns/op 1.11
LinkedList of 24000 items push then pop 6.4220 ns/op 6.4230 ns/op 1.00
intersect bitArray bitLen 8 4.9270 ns/op 4.6700 ns/op 1.06
intersect array and set length 8 30.282 ns/op 28.516 ns/op 1.06
intersect bitArray bitLen 128 24.434 ns/op 23.160 ns/op 1.06
intersect array and set length 128 507.20 ns/op 487.02 ns/op 1.04
bitArray.getTrueBitIndexes() bitLen 128 1.0590 us/op 1.0110 us/op 1.05
bitArray.getTrueBitIndexes() bitLen 248 1.8240 us/op 1.7400 us/op 1.05
bitArray.getTrueBitIndexes() bitLen 512 3.7300 us/op 3.5580 us/op 1.05
Full columns - reconstruct all 6 blobs 167.15 us/op 218.00 us/op 0.77
Full columns - reconstruct half of the blobs out of 6 75.603 us/op 104.45 us/op 0.72
Full columns - reconstruct single blob out of 6 31.131 us/op 41.269 us/op 0.75
Half columns - reconstruct all 6 blobs 384.40 ms/op 380.79 ms/op 1.01
Half columns - reconstruct half of the blobs out of 6 190.01 ms/op 190.13 ms/op 1.00
Half columns - reconstruct single blob out of 6 67.672 ms/op 68.187 ms/op 0.99
Full columns - reconstruct all 10 blobs 273.28 us/op 298.97 us/op 0.91
Full columns - reconstruct half of the blobs out of 10 134.41 us/op 199.80 us/op 0.67
Full columns - reconstruct single blob out of 10 31.760 us/op 60.783 us/op 0.52
Half columns - reconstruct all 10 blobs 664.14 ms/op 646.12 ms/op 1.03
Half columns - reconstruct half of the blobs out of 10 327.95 ms/op 325.10 ms/op 1.01
Half columns - reconstruct single blob out of 10 67.753 ms/op 71.936 ms/op 0.94
Full columns - reconstruct all 20 blobs 1.5123 ms/op 600.25 us/op 2.52
Full columns - reconstruct half of the blobs out of 20 182.83 us/op 342.49 us/op 0.53
Full columns - reconstruct single blob out of 20 29.014 us/op 30.670 us/op 0.95
Half columns - reconstruct all 20 blobs 1.2665 s/op 1.2549 s/op 1.01
Half columns - reconstruct half of the blobs out of 20 647.86 ms/op 632.71 ms/op 1.02
Half columns - reconstruct single blob out of 20 69.583 ms/op 66.834 ms/op 1.04
Set add up to 64 items then delete first 2.0843 us/op 1.9740 us/op 1.06
OrderedSet add up to 64 items then delete first 3.2997 us/op 3.1543 us/op 1.05
Set add up to 64 items then delete last 2.1124 us/op 2.0294 us/op 1.04
OrderedSet add up to 64 items then delete last 3.6456 us/op 3.2489 us/op 1.12
Set add up to 64 items then delete middle 2.5827 us/op 2.0964 us/op 1.23
OrderedSet add up to 64 items then delete middle 5.6265 us/op 4.7924 us/op 1.17
Set add up to 128 items then delete first 4.4057 us/op 4.1395 us/op 1.06
OrderedSet add up to 128 items then delete first 6.8027 us/op 5.8876 us/op 1.16
Set add up to 128 items then delete last 4.2770 us/op 3.9331 us/op 1.09
OrderedSet add up to 128 items then delete last 6.9134 us/op 7.6562 us/op 0.90
Set add up to 128 items then delete middle 4.8515 us/op 4.8757 us/op 1.00
OrderedSet add up to 128 items then delete middle 14.620 us/op 14.240 us/op 1.03
Set add up to 256 items then delete first 9.2509 us/op 9.3791 us/op 0.99
OrderedSet add up to 256 items then delete first 13.692 us/op 16.319 us/op 0.84
Set add up to 256 items then delete last 8.9046 us/op 10.262 us/op 0.87
OrderedSet add up to 256 items then delete last 14.469 us/op 14.088 us/op 1.03
Set add up to 256 items then delete middle 9.0376 us/op 8.0674 us/op 1.12
OrderedSet add up to 256 items then delete middle 42.060 us/op 39.805 us/op 1.06
pass gossip attestations to forkchoice per slot 2.6828 ms/op 2.4974 ms/op 1.07
forkChoice updateHead vc 100000 bc 64 eq 0 403.46 us/op 389.86 us/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 0 2.4794 ms/op 2.2227 ms/op 1.12
forkChoice updateHead vc 1000000 bc 64 eq 0 3.9534 ms/op 3.7068 ms/op 1.07
forkChoice updateHead vc 600000 bc 320 eq 0 2.3283 ms/op 2.2506 ms/op 1.03
forkChoice updateHead vc 600000 bc 1200 eq 0 2.4865 ms/op 2.3640 ms/op 1.05
forkChoice updateHead vc 600000 bc 7200 eq 0 3.8660 ms/op 3.3149 ms/op 1.17
forkChoice updateHead vc 600000 bc 64 eq 1000 3.0401 ms/op 2.7823 ms/op 1.09
forkChoice updateHead vc 600000 bc 64 eq 10000 3.1507 ms/op 3.0262 ms/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 300000 7.8718 ms/op 7.0111 ms/op 1.12
computeDeltas 1400000 validators 0% inactive 12.970 ms/op 12.515 ms/op 1.04
computeDeltas 1400000 validators 10% inactive 12.232 ms/op 11.812 ms/op 1.04
computeDeltas 1400000 validators 20% inactive 11.007 ms/op 10.596 ms/op 1.04
computeDeltas 1400000 validators 50% inactive 8.6916 ms/op 8.0472 ms/op 1.08
computeDeltas 2100000 validators 0% inactive 19.477 ms/op 18.706 ms/op 1.04
computeDeltas 2100000 validators 10% inactive 18.622 ms/op 17.803 ms/op 1.05
computeDeltas 2100000 validators 20% inactive 16.708 ms/op 16.165 ms/op 1.03
computeDeltas 2100000 validators 50% inactive 12.941 ms/op 9.5266 ms/op 1.36
altair processAttestation - 250000 vs - 7PWei normalcase 2.9898 ms/op 2.4686 ms/op 1.21
altair processAttestation - 250000 vs - 7PWei worstcase 3.5929 ms/op 3.5976 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 108.06 us/op 96.782 us/op 1.12
altair processAttestation - setStatus - 1/3 committees join 204.84 us/op 219.50 us/op 0.93
altair processAttestation - setStatus - 1/2 committees join 295.39 us/op 281.15 us/op 1.05
altair processAttestation - setStatus - 2/3 committees join 380.04 us/op 355.96 us/op 1.07
altair processAttestation - setStatus - 4/5 committees join 540.86 us/op 540.14 us/op 1.00
altair processAttestation - setStatus - 100% committees join 681.08 us/op 613.75 us/op 1.11
altair processBlock - 250000 vs - 7PWei normalcase 4.8825 ms/op 4.4057 ms/op 1.11
altair processBlock - 250000 vs - 7PWei normalcase hashState 15.733 ms/op 17.246 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase 23.619 ms/op 22.803 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 43.467 ms/op 42.375 ms/op 1.03
phase0 processBlock - 250000 vs - 7PWei normalcase 1.5847 ms/op 1.4336 ms/op 1.11
phase0 processBlock - 250000 vs - 7PWei worstcase 21.709 ms/op 17.429 ms/op 1.25
altair processEth1Data - 250000 vs - 7PWei normalcase 315.05 us/op 275.82 us/op 1.14
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:16 6.2160 us/op 3.0100 us/op 2.07
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:220 22.686 us/op 18.765 us/op 1.21
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:43 5.8660 us/op 5.6920 us/op 1.03
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:19 4.2690 us/op 4.4060 us/op 0.97
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1021 96.638 us/op 86.826 us/op 1.11
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11778 1.4049 ms/op 1.4322 ms/op 0.98
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.8755 ms/op 1.8114 ms/op 1.04
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8377 ms/op 1.7985 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 5.2363 ms/op 4.5213 ms/op 1.16
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.1165 ms/op 2.0123 ms/op 1.05
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.5913 ms/op 3.7384 ms/op 1.23
Tree 40 250000 create 373.02 ms/op 335.87 ms/op 1.11
Tree 40 250000 get(125000) 96.120 ns/op 88.860 ns/op 1.08
Tree 40 250000 set(125000) 1.0392 us/op 967.62 ns/op 1.07
Tree 40 250000 toArray() 19.300 ms/op 16.364 ms/op 1.18
Tree 40 250000 iterate all - toArray() + loop 17.350 ms/op 21.474 ms/op 0.81
Tree 40 250000 iterate all - get(i) 42.321 ms/op 42.052 ms/op 1.01
Array 250000 create 2.3416 ms/op 2.3758 ms/op 0.99
Array 250000 clone - spread 713.04 us/op 716.48 us/op 1.00
Array 250000 get(125000) 0.30500 ns/op 0.29900 ns/op 1.02
Array 250000 set(125000) 0.30400 ns/op 0.30400 ns/op 1.00
Array 250000 iterate all - loop 59.106 us/op 57.676 us/op 1.02
phase0 afterProcessEpoch - 250000 vs - 7PWei 46.122 ms/op 47.557 ms/op 0.97
Array.fill - length 1000000 2.3569 ms/op 2.9463 ms/op 0.80
Array push - length 1000000 9.9127 ms/op 10.147 ms/op 0.98
Array.get 0.21133 ns/op 0.20565 ns/op 1.03
Uint8Array.get 0.24475 ns/op 0.23343 ns/op 1.05
phase0 beforeProcessEpoch - 250000 vs - 7PWei 29.665 ms/op 17.140 ms/op 1.73
altair processEpoch - mainnet_e81889 326.79 ms/op 267.08 ms/op 1.22
mainnet_e81889 - altair beforeProcessEpoch 38.728 ms/op 18.222 ms/op 2.13
mainnet_e81889 - altair processJustificationAndFinalization 6.8790 us/op 7.1760 us/op 0.96
mainnet_e81889 - altair processInactivityUpdates 6.9550 ms/op 7.1201 ms/op 0.98
mainnet_e81889 - altair processRewardsAndPenalties 22.185 ms/op 22.439 ms/op 0.99
mainnet_e81889 - altair processRegistryUpdates 575.00 ns/op 550.00 ns/op 1.05
mainnet_e81889 - altair processSlashings 138.00 ns/op 146.00 ns/op 0.95
mainnet_e81889 - altair processEth1DataReset 136.00 ns/op 136.00 ns/op 1.00
mainnet_e81889 - altair processEffectiveBalanceUpdates 5.4254 ms/op 6.0572 ms/op 0.90
mainnet_e81889 - altair processSlashingsReset 718.00 ns/op 683.00 ns/op 1.05
mainnet_e81889 - altair processRandaoMixesReset 1.4980 us/op 1.3390 us/op 1.12
mainnet_e81889 - altair processHistoricalRootsUpdate 137.00 ns/op 132.00 ns/op 1.04
mainnet_e81889 - altair processParticipationFlagUpdates 477.00 ns/op 444.00 ns/op 1.07
mainnet_e81889 - altair processSyncCommitteeUpdates 111.00 ns/op 112.00 ns/op 0.99
mainnet_e81889 - altair afterProcessEpoch 42.932 ms/op 41.694 ms/op 1.03
capella processEpoch - mainnet_e217614 908.45 ms/op 764.40 ms/op 1.19
mainnet_e217614 - capella beforeProcessEpoch 59.453 ms/op 55.066 ms/op 1.08
mainnet_e217614 - capella processJustificationAndFinalization 6.6600 us/op 5.4190 us/op 1.23
mainnet_e217614 - capella processInactivityUpdates 14.952 ms/op 14.346 ms/op 1.04
mainnet_e217614 - capella processRewardsAndPenalties 88.725 ms/op 84.333 ms/op 1.05
mainnet_e217614 - capella processRegistryUpdates 4.5500 us/op 4.3540 us/op 1.05
mainnet_e217614 - capella processSlashings 134.00 ns/op 127.00 ns/op 1.06
mainnet_e217614 - capella processEth1DataReset 136.00 ns/op 127.00 ns/op 1.07
mainnet_e217614 - capella processEffectiveBalanceUpdates 24.671 ms/op 11.857 ms/op 2.08
mainnet_e217614 - capella processSlashingsReset 710.00 ns/op 653.00 ns/op 1.09
mainnet_e217614 - capella processRandaoMixesReset 1.2040 us/op 1.0880 us/op 1.11
mainnet_e217614 - capella processHistoricalRootsUpdate 134.00 ns/op 125.00 ns/op 1.07
mainnet_e217614 - capella processParticipationFlagUpdates 447.00 ns/op 408.00 ns/op 1.10
mainnet_e217614 - capella afterProcessEpoch 106.78 ms/op 105.29 ms/op 1.01
phase0 processEpoch - mainnet_e58758 309.94 ms/op 289.13 ms/op 1.07
mainnet_e58758 - phase0 beforeProcessEpoch 65.327 ms/op 60.324 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 6.3520 us/op 5.6000 us/op 1.13
mainnet_e58758 - phase0 processRewardsAndPenalties 15.947 ms/op 15.269 ms/op 1.04
mainnet_e58758 - phase0 processRegistryUpdates 2.2960 us/op 2.1750 us/op 1.06
mainnet_e58758 - phase0 processSlashings 134.00 ns/op 130.00 ns/op 1.03
mainnet_e58758 - phase0 processEth1DataReset 266.00 ns/op 259.00 ns/op 1.03
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.6389 ms/op 800.18 us/op 3.30
mainnet_e58758 - phase0 processSlashingsReset 960.00 ns/op 822.00 ns/op 1.17
mainnet_e58758 - phase0 processRandaoMixesReset 1.4140 us/op 1.1910 us/op 1.19
mainnet_e58758 - phase0 processHistoricalRootsUpdate 143.00 ns/op 126.00 ns/op 1.13
mainnet_e58758 - phase0 processParticipationRecordUpdates 1.2190 us/op 951.00 ns/op 1.28
mainnet_e58758 - phase0 afterProcessEpoch 33.869 ms/op 32.005 ms/op 1.06
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0174 ms/op 947.75 us/op 1.07
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.5136 ms/op 1.9485 ms/op 1.80
altair processInactivityUpdates - 250000 normalcase 13.199 ms/op 10.327 ms/op 1.28
altair processInactivityUpdates - 250000 worstcase 13.713 ms/op 10.192 ms/op 1.35
phase0 processRegistryUpdates - 250000 normalcase 2.4560 us/op 2.1810 us/op 1.13
phase0 processRegistryUpdates - 250000 badcase_full_deposits 141.20 us/op 130.04 us/op 1.09
phase0 processRegistryUpdates - 250000 worstcase 0.5 65.825 ms/op 59.517 ms/op 1.11
altair processRewardsAndPenalties - 250000 normalcase 15.966 ms/op 16.003 ms/op 1.00
altair processRewardsAndPenalties - 250000 worstcase 16.355 ms/op 15.256 ms/op 1.07
phase0 getAttestationDeltas - 250000 normalcase 7.7238 ms/op 5.0876 ms/op 1.52
phase0 getAttestationDeltas - 250000 worstcase 11.429 ms/op 7.9911 ms/op 1.43
phase0 processSlashings - 250000 worstcase 62.305 us/op 56.276 us/op 1.11
altair processSyncCommitteeUpdates - 250000 11.570 ms/op 9.5739 ms/op 1.21
BeaconState.hashTreeRoot - No change 172.00 ns/op 172.00 ns/op 1.00
BeaconState.hashTreeRoot - 1 full validator 86.779 us/op 78.398 us/op 1.11
BeaconState.hashTreeRoot - 32 full validator 878.50 us/op 819.77 us/op 1.07
BeaconState.hashTreeRoot - 512 full validator 9.0960 ms/op 8.7617 ms/op 1.04
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 113.29 us/op 95.440 us/op 1.19
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.4809 ms/op 1.3696 ms/op 1.08
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.302 ms/op 17.029 ms/op 1.19
BeaconState.hashTreeRoot - 1 balances 88.288 us/op 70.368 us/op 1.25
BeaconState.hashTreeRoot - 32 balances 800.06 us/op 700.26 us/op 1.14
BeaconState.hashTreeRoot - 512 balances 7.0952 ms/op 5.9649 ms/op 1.19
BeaconState.hashTreeRoot - 250000 balances 136.07 ms/op 126.22 ms/op 1.08
aggregationBits - 2048 els - zipIndexesInBitList 19.715 us/op 18.314 us/op 1.08
regular array get 100000 times 23.117 us/op 21.951 us/op 1.05
wrappedArray get 100000 times 23.304 us/op 22.112 us/op 1.05
arrayWithProxy get 100000 times 10.257 ms/op 16.791 ms/op 0.61
ssz.Root.equals 21.654 ns/op 20.547 ns/op 1.05
byteArrayEquals 21.450 ns/op 20.422 ns/op 1.05
Buffer.compare 8.9300 ns/op 8.4600 ns/op 1.06
processSlot - 1 slots 10.348 us/op 8.6800 us/op 1.19
processSlot - 32 slots 2.1896 ms/op 1.8763 ms/op 1.17
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 3.5847 ms/op 3.5573 ms/op 1.01
getCommitteeAssignments - req 1 vs - 250000 vc 1.6312 ms/op 1.5898 ms/op 1.03
getCommitteeAssignments - req 100 vs - 250000 vc 3.3718 ms/op 3.2733 ms/op 1.03
getCommitteeAssignments - req 1000 vs - 250000 vc 3.6045 ms/op 3.4983 ms/op 1.03
findModifiedValidators - 10000 modified validators 733.80 ms/op 671.22 ms/op 1.09
findModifiedValidators - 1000 modified validators 523.21 ms/op 392.96 ms/op 1.33
findModifiedValidators - 100 modified validators 290.99 ms/op 275.43 ms/op 1.06
findModifiedValidators - 10 modified validators 232.99 ms/op 195.69 ms/op 1.19
findModifiedValidators - 1 modified validators 127.33 ms/op 141.65 ms/op 0.90
findModifiedValidators - no difference 150.60 ms/op 152.58 ms/op 0.99
migrate state 1500000 validators, 3400 modified, 2000 new 3.2050 s/op 2.6225 s/op 1.22
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 3.8000 ns/op 3.6400 ns/op 1.04
state getBlockRootAtSlot - 250000 vs - 7PWei 427.00 ns/op 330.75 ns/op 1.29
computeProposerIndex 100000 validators 1.4170 ms/op 1.2639 ms/op 1.12
getNextSyncCommitteeIndices 1000 validators 2.9468 ms/op 2.7579 ms/op 1.07
getNextSyncCommitteeIndices 10000 validators 26.154 ms/op 24.135 ms/op 1.08
getNextSyncCommitteeIndices 100000 validators 93.251 ms/op 79.462 ms/op 1.17
computeProposers - vc 250000 575.98 us/op 552.18 us/op 1.04
computeEpochShuffling - vc 250000 40.903 ms/op 38.116 ms/op 1.07
getNextSyncCommittee - vc 250000 12.078 ms/op 9.1994 ms/op 1.31
nodejs block root to RootHex using toHex 94.814 ns/op 84.136 ns/op 1.13
nodejs block root to RootHex using toRootHex 54.773 ns/op 49.338 ns/op 1.11
nodejs fromHex(blob) 1.0713 ms/op 680.49 us/op 1.57
nodejs fromHexInto(blob) 631.89 us/op 599.34 us/op 1.05
nodejs block root to RootHex using the deprecated toHexString 486.40 ns/op 442.91 ns/op 1.10
nodejs byteArrayEquals 32 bytes (block root) 25.942 ns/op 24.900 ns/op 1.04
nodejs byteArrayEquals 48 bytes (pubkey) 38.107 ns/op 35.695 ns/op 1.07
nodejs byteArrayEquals 96 bytes (signature) 34.167 ns/op 31.936 ns/op 1.07
nodejs byteArrayEquals 1024 bytes 41.089 ns/op 39.579 ns/op 1.04
nodejs byteArrayEquals 131072 bytes (blob) 1.8210 us/op 1.6856 us/op 1.08
browser block root to RootHex using toHex 154.53 ns/op 137.62 ns/op 1.12
browser block root to RootHex using toRootHex 134.26 ns/op 124.72 ns/op 1.08
browser fromHex(blob) 1.8290 ms/op 1.4143 ms/op 1.29
browser fromHexInto(blob) 653.49 us/op 598.15 us/op 1.09
browser block root to RootHex using the deprecated toHexString 363.74 ns/op 312.36 ns/op 1.16
browser byteArrayEquals 32 bytes (block root) 29.182 ns/op 26.673 ns/op 1.09
browser byteArrayEquals 48 bytes (pubkey) 41.303 ns/op 37.484 ns/op 1.10
browser byteArrayEquals 96 bytes (signature) 77.196 ns/op 70.744 ns/op 1.09
browser byteArrayEquals 1024 bytes 792.27 ns/op 724.60 ns/op 1.09
browser byteArrayEquals 131072 bytes (blob) 101.13 us/op 90.996 us/op 1.11

by benchmarkbot/action

Copy link
Copy Markdown
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • the fix of #8150 was to throw error if there is 0-block returned, this is incorrect in case the network really has 0 block, especially in devnets. But it rarely happens and the fix helped us to be able to range sync on fulu devnets at the time
  • this fix handles 0-block returned, but there could be issue in regular networks where peers did not return correctly, at least I know lodestar did not handle it well
    chain.logger.verbose("Peer did not respect earliestAvailableSlot for BeaconBlocksByRange", {

the problem with #8147 is that node downloaded multiple empty block epoch and got stuck
it happened there in fulu devnets because earliestAvailableSlot was not fully implemented by clients

the change seems good to me but we need to make sure we can range sync on the next glamsterdam-devnet-5 + hoodi + mainnet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants